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ALDs Role in Score Reporting 


• Communicate what scores mean 

• Communicate what examinees at different 
performance levels know and are able to do 

• Descriptions of performance level communicated 
through the ALDs 
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ECD & ALDs 


• Through the domain analysis & domain modeling, 
claims and evidence pairs are generated 

• These claims and evidence pairs and evidence 
pairs are then mapped onto the performance 
continuum 

• Focus is on exemplar claims that differentiate 
performance levels 
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Articulation of ALDs 


• Using exemplar claims SMEs then put into words 
the defining characteristics of the performance 
level to generate preliminary ALDs 

• Iterative process 

• Identification of gaps in current set of claims 

• New claims may identify need to refine ALDs 
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Benefits of Creating ALDs with ECD 


• Continues the direct connection to intended score 
interpretations (through claims and evidence 
pairs) 

• Reinforces the need to have tasks developed and 
included in the assessment that represent the 
ALD levels of performance - both in item 
development and form assembly specifications 

• Strengthens the trail of validity evidence to 
support score interpretation and use 
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ALD Methodology 


Illustrative study 

• History (World, European, US) 

• SMEs from the committees who worked on the domain 
analysis and generated claims and evidence pairs 
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General Procedures 


• Work in subject-specific groups 

• Map illustrative claims (typically 3) from the major 
domain topics identified from the domain analysis to 
score levels 3, 4, 5 

• Synthesize these claims into ALDs for score levels 3, 4 


• Discipline-level ALDs 

• Look for similarities across subject-specific ALD 

• Articulate discipline-level ALDs based on commonalities 
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History ALDs: ECD Input 


• Domain analysis organized content primarily by 
time periods 

• Claims and evidence pairs from domain modeliing 

• Draft Historical Thinking Skills 
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History: Subject-specific 
Groups 

• Orientation included modeling of how a claim could 
be modified (e.g., through changing the HTS) to 
change its location on the score continuum 

• Each group was tasked with mapping (or modifying) 
exemplar claims onto performance continuum 

• Forms were provided for recording exemplar claims 

• Each group chose own strategy; task to be 
completed by end of day 
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History: Generation of Draft ALDs 


• Subject-specific groups: morning task 

• Bulleted lists/tables 

• Preparation of presentation 

• Discipline ALD 

• Afternoon 

• Each group presentation 

• AP History coordinator summarized, looked for trends 
and commonalities 
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Results 


• Generated less than fully articulated subject- 
specific ALDs or discipline ALDs 

• Generated bulleted lists 

• Identified “difficulty drivers” 

• Needed more time to process similarities 
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Iterative Process 


• Expected that subject-specific and discipline-level 
ALDs would not be fully articulated 

• Study was first step in generating initial ALDs 

• These were then used to develop more claims and 
evidence pairs for domain modeling 

• Symbiotic relationship through iterations 

• Claims informed initial ALDs 

• ALDs were refined by more development of claims 


CoUegeBoard 

inspiring minds' 


History (11) 


Orientation clear 

1 00% 

Orientation helpful 

1 00% 

Understood purpose 

1 00% 

Understood task 

82% 

Adj perf cats 

73% 

Mapping for ALDs 

90% 

Consensus 

90% 

Time 

82% 

Role in ALDs 

1 00% 

Subject-specific ALD 

90% 


Conclusion 


• ECD supports intended score interpretations 
through 

• Domain analysis & modeling -- claims and evidence 
pairs 

• Claims and evidence pairs are mapped to score scale -- 
create ALDs 

• ALDs helps inform task development & form assembly 
specifications 

• ALDs and examinee performance can inform cutscores 
& provide validity evidence 
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